Skip to content

V0.18.0 fix#22

Merged
NickLucche merged 3 commits into
masterfrom
v0.18.0-fix
Apr 30, 2026
Merged

V0.18.0 fix#22
NickLucche merged 3 commits into
masterfrom
v0.18.0-fix

Conversation

@NickLucche

Copy link
Copy Markdown
Member

Fixes for bart based on #20, ensuring same output as v0.16.0

python example_bart_usage.py

output:  Trump is president of the United States. The president of the United States is president of the United States
output:  the city of Paris, the city of Paris is the city of the city of Paris. The city

dschulmeist and others added 3 commits April 16, 2026 11:32
The multimodal processor in bart.py broke under vLLM 0.18 in three places:

- TextDataParser relied on MultiModalDataParser._is_empty, which was
  removed in 0.18. Replaced with inline emptiness checks for str and list.

- create_encoder_prompt previously tokenized `prompt` as the encoder text.
  In 0.18 `inputs.prompt` passed to this method is the DECODER prompt text,
  not the encoder text (the encoder content lives in mm_data). The method
  now returns a single [0] placeholder token; _get_prompt_updates replaces
  it with the correct number of encoder token slots during rendering.

- _call_hf_processor is now sometimes called with an already-tokenized
  decoder prompt (list[int]) instead of a str. Handle both cases when
  building result["input_ids"].

Adds tests/test_vllm_018_compat.py with three narrow unit tests covering
each of these paths; no GPU required.

Signed-off-by: David Schulmeister <dschulmeist@users.noreply.github.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
Signed-off-by: NickLucche <nlucches@redhat.com>
@NickLucche NickLucche merged commit bab97cd into master Apr 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants